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Abstract 

Recently,  a  C4ISR  Information  Superiority  Modeling  &  Simulation  Master  Plan  (IS  M&S  MP) 
was  issued  by  the  Office  of  the  Secretary  of  Defense  (OSD)  addressing  the  application  of  M&S 
to  all  functional  areas  associated  with  C4ISR  (e.g.,  training,  support  to  operations,  acquisition) 
(Reference  1).  In  the  area  of  acquisition,  the  C4ISR  IS  M&S  MP  called  for  the  following  actions: 
recommend  and  outline  methods  for  providing  improved  M&S  support  to  C4ISR  acquisition; 
and  provide  recommendations  of  desirable  policy  changes  and  initiatives,  along  with  metrics. 

To  implement  those  actions,  a  workshop  was  convened  on  2  -  4  April  2002  under  the 
sponsorship  of  the  Office  of  the  Assistant  Secretary  of  Defense  (C3I);  the  Director, 
Interoperability,  Undersecretary  of  Defense  (Acquisition,  Technology,  &  Logistics);  and  the 
Director,  Defense  Modeling  &  Simulation  Office  (DMSO).  Forty  four  subject  matter  experts 
from  government,  industry,  and  academia  were  assembled  to  address  the  subject. 

The  participants  were  organized  into  breakout  groups  to  address  C4ISR  acquisition  by  system 
type  (sensors,  communications,  C2/information  processing)  and  systems-of-sy stems.  In  addition, 
a  synthesis  panel  was  convened  to  identify  and  summarize  the  similarities  and  differences  in  the 
findings  and  recommendations  of  the  other  breakout  groups.  To  initiate  the  deliberations,  the 
breakout  groups  were  provided  with  a  strawman  list  of  M&S  capability  objectives  to  support 
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C4ISR  acquisition.  For  each  of  the  major  phases  in  the  Department  of  Defense  Directive  5000  on 
the  acquisition  of  systems  (Reference  2),  the  breakout  groups  used  the  strawman  list  of  M&S 
capability  objectives  to  identify  and  prioritize  key  M&S  needs  (i.e.,  the  cases  where  critical 
M&S  capability  objectives  could  not  be  satisfied  by  the  year  2007).  Subsequently,  each  breakout 
group  looked  across  the  high  priority  needs  identified  in  each  phase  of  the  acquisition  process 
and  formulated  options  to  ameliorate  the  most  critical  of  the  M&S  needs.  The  synthesis  panel 
completed  the  deliberations  by  identifying  overall,  critical  M&S  needs  and  proposed  investment 
options. 

This  paper  summarizes  the  highest  priority  M&S  needs  that  were  identified  to  support  the 
acquisition  and  transformation  of  C4ISR  systems  over  the  next  five  years.  It  is  anticipated  that 
the  findings  and  recommendations  from  this  workshop  will  drive  the  Department  of  Defense’s 
investment  in  M&S  to  support  the  acquisition  and  transformation  of  C4ISR  systems. 

A.  Introduction 

This  paper  is  divided  into  five  sections.  The  first  section,  the  Context,  describes  the  background 
for  the  workshop,  identifies  the  goal,  objectives,  and  scope  of  the  activity.  The  second  section 
sets  the  stage  for  the  deliberations  of  the  Panel  by  describing  briefly  how  the  C4ISR  acquisition 
problem  has  changed  since  the  end  of  the  Cold  War.  The  third  section  summarizes  the  insights 
and  challenges  that  were  derived  from  the  remarks  of  the  individuals  that  spoke  during  the 
introductory  plenary  session.  These  individuals  can  be  aggregated  into  three  categories:  the 
sponsors  of  the  workshop,  representatives  that  described  key  institutional  products,  and  Service 
representatives  that  discussed  major  initiatives  in  the  area  of  M&S  environments  to  support  the 
acquisition  of  C4ISR.  The  fourth  section  establishes  a  framework  for  M&S  to  support  the 
acquisition  and  transformation  of  C4ISR  and  embeds  the  major  findings  of  the  individual  panels 
in  that  framework.  That  gives  rise  to  insights  on  the  similarities  and  differences  in  the  panels’ 
findings  and  recommendations.  The  report  concludes  with  a  Summary  that  describes  what  the 
workshop  accomplished  and  residual  challenges  that  remain  to  be  addressed. 

B.  Context 

The  underlying  rationale  for  this  workshop  emerged  from  the  recently  released  Information 
Superiority  Modeling  &  Simulation  Master  Plan  (IS  M&S  MP)  (Reference  1).  That  plan 
addresses  the  application  of  M&S  to  all  functional  areas  (e.g.,  training,  support  to  operations, 
acquisition).  Consistent  with  those  interests,  the  goal  of  the  workshop  was  to  provide  input  to  an 
investment  plan  for  M&S  to  support  the  acquisition  and  transformation  of  C4ISR  systems. 

In  order  to  support  that  goal,  this  paper  addresses  three  objectives.  First  it  captures  relevant 
insights  from  the  initial  plenary  presentations.  Second,  it  identifies  and  summarizes  the 
similarities  and  differences  in  the  findings  and  recommendations  of  the  workshop  break-out 
groups.  Finally,  it  summarizes  the  highest  priority  M&S  needs  that  emerged  from  the  workshop. 

Although  the  workshop  had  relatively  broad  goals  and  objectives,  it  is  important  to  highlight  key 
areas  that  it  did  not  address.  First,  it  did  not  analyze  the  M&S  needs  associated  with  pre¬ 
acquisition  activities  (e.g.,  requirements  analyses).  These  activities  are  outside  the  responsibility 


of  the  sponsors  of  this  workshop.  In  addition,  the  workshop  did  not  address  systems  that 
transcended  the  control  of  the  Department  of  Defense  (DoD)  or  the  Intelligence  Community 
(IC).  Consequently,  it  did  not  address  the  issues  associated  with  C4ISR  systems  that  include 
inter-agency  mission/systems  (e.g.,  the  systems  of  the  Federal  Emergency  Management  Agency, 
Department  of  Justice)  even  though  these  systems  are  likely  to  be  of  increasing  interest  in  the 
evolving  mission  of  homeland  security.  In  addition,  it  did  not  address  specifically  the  issues 
associated  with  coalition/alliance  operations.  These  are  significant  shortcomings  that  should  be 
addressed  in  future  workshops. 

Consistent  with  these  goals,  objectives,  and  scope,  the  workshop  employed  the  following 
approach.  To  establish  a  common  framework,  the  workshop  began  with  a  plenary  session  that 
featured  presentations  from  the  workshop  sponsors,  briefings  on  key  institutional  products  and 
processes,  and  presentations  on  major  Service  initiatives.  Subsequently,  the  participants  were 
divided  into  several  break-out  groups:  single  system  panels  (i.e.,  sensors,  communications, 
command  and  control/information  processing)  and  a  system-of-systems  panel.  In  addition,  a 
synthesis  panel  was  established  to  provide  a  broad,  cross-cutting  perspective  on  the  subject. 

At  the  outset  of  the  break-out  phase  of  the  workshop,  the  individual  panels  were  requested  to 
begin  with  strawman  M&S  capability  objectives  and  refine  them  (e.g.,  add  or  modify  elements  of 
the  list).  By  comparing  these  capability  objectives  to  projected  capabilities  for  each  phase  in  the 
acquisition  process  (in  the  2007  timeframe),  the  panels  were  to  identify  residual  needs  and  then 
to  select  the  highest  priority  needs.  Subsequently,  the  three  to  five  highest  priority  needs,  overall, 
were  to  be  culled  out  and  potential  solutions  provided. 

C.  Nature  of  the  C4ISR  Acquisition  Problem 


This  section  of  the  paper  briefly  characterizes  the  nature  of  the  evolving  C4ISR  acquisition 
process.  It  describes  the  new  DoD  acquisition  context  and  associated  DoD  C4ISR  acquisition 
challenges. 


Old  L 

)  New 

•  Well  understood  threat 

•  New  and  uncertain  threats 

•  Established  scenarios/operations 

•  Broad  range  of  missions 

•  DOD  focus 

•  National,  coalition  perspective 

•  Evolutionary  capability 

•  Revolutionary  capability 

•  Force-on-force 

•  Info/Effects-based  outcome 

•  System-on-system  advantage 

•  System-of-Systems  advantage 

•  Requirements  derived  from  deficiencies 

•  Exploration/Learning 

Table  1.  A  New  DOD  Acquisition  Context 


Today’s  acquirers  find  themselves  in  a  new  national  security  context.  Table  1  highlights  some  of 
the  dramatic  shifts  that  have  occurred  since  the  end  of  the  Cold  War.  Having  the  Soviet  Union 
as  the  single  dominant  adversary  over  a  protracted  period  provided  a  sustained  focus  for 
intelligence  gatherers  and  force  planners  so  that  they  could  refine  U.S.  understanding  of  many 
aspects  of  Soviet  capability  and  behavior.  This  is  in  stark  contrast  to  the  “New  World  Disorder” 


in  which  a  broad  range  of  varied  and  uncertain  threats  have  made  it  difficult  to  anticipate  issues 
and  focus  intelligence  resources  appropriately. 

As  a  consequence  of  the  Soviet  focus,  a  relatively  few  scenarios  and  types  of  operations  were 
sufficient  for  assessment  and  planning.  Today  the  U.S.  is  faced  with  a  broad  set  of  operations 
and  missions  that  include  homeland  security,  peace  keeping,  coercive  operations,  small-scale 
contingencies,  and  major  theater  conflicts. 

With  the  exception  of  the  NATO  allies,  DoD  was  mainly  concerned  with  operations  that 
involved  only  the  four  Services.  Many  operations  today  require  a  much  larger  contingent  of 
participants,  including  numerous  non-NATO  allies,  various  national  government  organizations, 
International  Organizations  (IOs),  and  Non-governmental  Organizations  (NGOs). 

Until  now,  war  fighting  capability  has  evolved  incrementally  with  the  addition  of  each  new 
weapon  system.  However,  information  technology  and  precision  weaponry  has  the  potential  of 
changing  the  nature  of  warfare  in  revolutionary  ways. 

In  the  past,  the  key  to  success  was  believed  to  be  determined  by  who  could  bring  to  bear 
overwhelming  force.  Today,  the  goal  is  becoming  one  of  exploiting  information  about  U.S. 
adversaries  to  apply  the  minimum  force  in  order  to  achieve  specific  effects,  consistent  with 
national  policy. 

Advantage  was  often  measured  in  platform-centric  terms  (e.g.,  who  had  the  best  tank,  ship,  or 
plane).  Today,  when  the  sensors,  C2,  and  weapons  are  networked  together  in  a  system-of- 
systems,  they  promise  significant  advantage  through  increased  agility  and  discriminate 
apphcation. 

Finally,  the  stable,  evolutionary  environment  in  which  requirements  are  relatively  well 
understood  have  given  way  to  a  period  of  experimentation  and  learning  necessary  to  understand 
how  to  exploit  rapidly  emerging  technologies  and  new  concepts  in  order  to  maintain  a 
competitive  advantage.  Taken  together,  these  shifts  add  up  to  a  fundamentally  different  national 
security  context  within  which  today’s  acquirers  must  function. 


Old  1 

New 

•  Threat  based 

•  Capability  based 

•  Refining  established  systems 

•  Exploring  new  possibilities 

•  Benefits  of  new  capabilities 

•  Understanding  fundamentals 

•  Firepower-centric 

•  Speed-centric 

•  Acquiring  force  structure 

•  Mission  capability  packages  (e.g., 

DOTML-PF) 

•  Collection  of  ad  hoc  issues 

•  Hierarchy  of  related  issues 

•  Tractable  focus 

•  Exploding  complexity 

Table  2.  New  DOD  C4ISR  Acquisition  Challenges 


Shifts  in  the  national  security  context  have  resulted  in  major  changes  in  the  challenges  faced  by 
teams  of  individuals  that  seek  to  acquire  effective  C4ISR  systems  rapidly  and  efficiently.  Some 
of  the  key  changes  are  highlighted  in  Table  2. 

In  the  old  context,  acquirers  could  focus  on  means  of  countering  a  specific  threat.  Today  they 
must  address  capabilities  that  can  be  used  in  an  agile  manner  to  deal  with  a  range  of  threats. 
Similarly,  because  of  the  stability  of  the  threat  and  the  evolutionary  nature  of  military  capability, 
acquirers  used  to  depend  upon  established  operational  concepts  and  capabilities.  This  contrasts 
with  the  current  challenge  where  acquirers  must  cope  with  completely  new  war  fighting  concepts 
like  distributed  C2  for  the  nonlinear  battlespace. 

In  the  past,  acquirers  could  focus  on  the  benefits  of  adding  a  new  C4ISR  system  to  the  force  mix. 
Today,  they  must  understand  the  fundamentals  associated  with  networking  the  force  or  sharing 
information  through  a  common  operational  picture.  In  addition,  assessments  used  to  be  focused 
on  force  mix/structure  issues.  Today,  assessments  must  address  new  mission  capability  packages, 
including  all  the  dimensions  of  doctrine,  organization,  training,  materiel,  leadership  &  education, 
personnel,  and  facilities  (DOTML-PF).  Previously,  acquirers  often  focused  on  issues  in  isolation 
(e.g.,  for  communications  systems,  maximizing  bandwidth  for  specified  cost)  .  Today  there  is  a 
need  for  a  systematic  multi-level  assessment  of  a  comprehensive  set  of  related  issues  in  a 
system-of-systems  context.  Finally,  the  expanded  dimensionality  reflected  in  the  above  factors 
combined  with  the  increase  in  the  number  of  players  translates  into  significant  growth  in  problem 
complexity. 

D.  Plenary  Insights,  Challenges 

This  third  section  of  the  paper  summarizes  the  insights  and  challenges  that  were  derived  from  the 
presentations  during  the  initial  plenary  session.  The  plenary  session  began  with  presentations  by 
the  workshop  sponsors  or  their  representatives:  Director,  Interoperability,  Undersecretary  of 
Defense  (Acquisition,  Technology,  &  Logistics);  Assistant  Secretary  of  Defense  (Command, 
Control,  Communications,  and  Intelligence);  and  Director,  Defense  Modeling  &  Simulation 
Office.  They  were  followed  by  individuals  who  described  and  discussed  key  institutional 
products  and  processes:  the  IS  M&S  MP  and  Department  of  Defense  Directive  (DoDD)  5000. 
The  plenary  concluded  with  presentations  discussing  innovations  that  the  Services  are  pursuing 
in  the  area  of  M&S  to  support  the  acquisition  and  transformation  of  C4ISR  systems:  the  United 
States  Air  Force  (USAF)  Joint  Synthetic  Battlespace  (JSB),  the  United  States  Navy  (USN) 
Collaborative  Engineering  Environment,  and  the  United  States  Army  (USA)  evolving  Future 
Combat  System  (FCS)  and  the  Joint  Virtual  Battlespace  (JVB). 

Over  all,  a  common  perspective  emerged  from  the  remarks  of  the  sponsors  and  their 
representatives.  All  see  M&S  as  critical  to  their  organization’s  mission  effectiveness.  With 
respect  to  systems-of-systems,  all  saw  it  as  a  wave  of  the  future  which  pose  new  M&S 
challenges  for  the  community. 

The  presentation  on  the  IS  M&S  MP  emphasized  its  four  major  goals.  These  include: 

•  Assess  the  value  of  information  superiority; 

•  Enhance  the  integration  of  M&S  with  real  C4ISR  systems; 


•  Develop  M&S  that  reflect  the  decisionmaking  and  behavior  of  Blue  and  Red  force 
commanders;  and 

•  Improve  M&S  for  acquisition  and  transformation. 

Clearly,  it  is  this  latter  goal  that  was  the  main  focus  of  this  workshop. 


The  5000  Model 


Process  entry  at  Milestones  A 
8  or  C  (or  within  phases) 

‘Entrance  criteria'  met  before 
entering  phase 
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Figure  1.  The  5000  Model 


The  presentation  on  DoD  acquisition  directives  began  by  citing  shortfalls  in  the  existing 
acquisition  process  (e.g.,  it  takes  too  long,  costs  too  much,  and  is  incompatible  with  modern 
subsystem  cycle  times).  The  opportunities  offered  by  the  acquisition  model  described  in  the 
latest  version  of  DoDD  5000  were  cited  (see  Figure  1).  It  was  stressed  that  the  model  emphasizes 
the  use  of  evolutionary  acquisition,  anticipating  the  need  to  inject  new  technology  periodically 
(e.g.,  on  the  order  of  every  eighteen  months  to  keep  pace  with  developments  in  the  information 
systems  sector  of  the  commercial  world).  Several  enhanced  roles  for  M&S  in  support  of  this  new 
model  were  identified.  These  included  support  to  Analyses  of  Alternatives  (AoAs)  in  Concept  & 
Technology  Development  (CDT),  Early  Operational  Assessments  (EOAs)  in  System 
Development  &  Demonstration  (SDD),  and  Operational  Test  &  Evaluation  (OT&E)  in 
Production  &  Deployment  (P&D). 

However,  it  was  acknowledged  that  there  are  several  major  residual  challenges  in  DoDD  5000 
that  require  future  action,  particularly  in  the  guidance  it  provides  on  system-of-systems 


acquisition.  For  example,  since  individual  systems  are  acquired  asynchronously,  how  do  you  do 
“full  system  demonstration”  before  the  commitment  to  production  or  perform  T&E  of  a  full 
system-of-systems?  In  addition  how  do  you  deal  with  interoperability  and  supportability  of  a 
C4ISR  system  when  multiple  configurations  are  deployed,  simultaneously?  This  latter  issue  is  of 
particular  concern  to  the  US  Army  that  must  deal  with  the  simultaneous  fielding  of  large 
numbers  of  legacy,  interim,  and  objective  systems. 

In  the  final  segment  of  the  plenary,  representatives  from  the  three  Services  described  key 
initiatives.  The  Electronic  Systems  Command,  USAF,  is  developing  a  JSB  to  support  the  future 
acquisition  and  integration  of  C4ISR-weapon  systems-of-systems.  This  effort  has  pursued  a  “top 
down”  approach”,  systematically  addressing  the  dimensions  of  leadership,  policy,  process, 
technology,  and  resources.  To  take  advantage  of  the  extensive  investment  in  M&S  environments 
that  have  been  made,  this  approach  strongly  emphasizes  Service/industry  collaboration.  Two 
residual  issues  were  highlighted.  First,  today’s  M&S  capabilities  do  not  adequately  portray  the 
complexity  of  the  proposed  USAF  Global  Strike  Task  Force.  How  can  we  enhance  fidelity  of 
M&S  to  ameliorate  this  shortfall?  Second,  the  JSB  staff  participated  in  the  April  2002  Joint 
Combat  Identification  Evaluation  Team  (JCIET)  in  order  to  acquire  data  to  support  JSB’s 
Verification  and  Validation  (V&V)  activities.  However,  it  is  not  clear  how  one  can 
systematically  perform  V&V  of  existing  M&S  and  environments  from  the  data  that  were 
collected.  A  question  was  also  raised  about  who  will  own  the  JCIET  data. 

Second,  the  USN  is  developing  a  Naval  Collaborative  Engineering  Environment  to  support 
Program  Executive  Officers  (PEOs)  and  Program  Managers  (PMs)  in  meeting  the  Service’s 
integration  and  interoperability  requirements.  They  have  adopted  a  framework  for  integrated 
system  evolution  that  emphasizes  the  separation  of  data  and  a  suite  of  multi-functional  tools  to 
support  the  initial  phase  of  acquisition  (e.g.,  mission  capability  packages  supported  by 
Operational  Requirement  Documents,  Analyses  of  Alternatives).  Using  a  “best  of  breed” 
philosophy,  they  have  assembled  tools  to  support  processes  such  as  mission  analysis  (e.g..  Navy 
Simulation  System),  architectural  representation  (e.g.,  Rational  Rose),  requirements  analysis 
(e.g.,  DOOR),  and  functional  analysis  (e.g.,  CORE).  Among  the  residual  issues  are  the 
configuration  control  and  management  of  the  data  base,  the  expansion  of  the  tool  set  to  meet 
emerging  needs,  and  the  evolution  of  the  Collaborative  Engineering  Environment  to  support  later 
stages  of  the  acquisition. 

Finally,  the  USA  completed  the  Service  perspectives  by  describing  current  thoughts  on  the 
C4ISR-weapon  system  mix  for  the  proposed  FCS  and  the  tools  that  are  evolving  to  support  the 
acquisition  of  that  capability.  In  the  latter  area,  they  are  creating  an  arena  to  evaluate  future 
system  concepts  drawing  on  the  capabilities  developed  at  the  Joint  Precision  Strike 
Demonstration  (JPSD)  and  the  JVB.  The  philosophy  is  to  integrate  existing  M&S  tools  and 
resources  from  a  variety  of  sources  (e.g.,  USA  Research,  Development,  and  Engineering  Centers 
(RDECs),  National  Labs)  and  to  augment  them  via  a  spiral  build  to  address  specific  system 
acquisitions. 

Two  residual  issues  were  highlighted  for  FCS.  First,  the  need  for  enhanced  human  performance 
modeling  was  emphasized,  particularly  at  the  commander/staff  level.  In  addition,  concern  was 
raised  about  the  ability  of  current  tools  to  address  future  system  concepts.  In  particular,  the 


challenges  associated  with  modeling  future  sensor  management  concepts  and  overall 
communications  networking  were  identified. 

Looking  across  the  presentations  by  the  three  Services,  it  is  noteworthy  that  cross-Service 
collaboration  on  their  activities  is  ongoing  and  increasing.  Since  each  Service  has  employed  a 
different,  but  complementary  approach,  to  the  issue,  such  collaboration  has  the  potential  to 
enhance  the  quality  of  each  Service’s  environments  and  to  minimize  duplication  of  effort. 

E.  Panel  Findings  and  Recommendations 

This  section  of  the  paper  addresses  the  break-out  panels’  findings  and  recommendations.  It 
begins  by  identifying  and  discussing  insights  that  were  raised  in  the  areas  of  systems-of-systems 
and  cultural  barriers.  It  then  establishes  a  synthesizing  framework  to  capture,  compare,  and 
contrast  the  findings  and  recommendations  of  the  workshop  panels. 

During  the  course  of  the  workshop  numerous  discussions  occurred  about  the  challenges 
associated  with  acquiring  systems-of-systems.  It  was  observed  that  in  the  absence  of  a  system-of- 
systems  context,  we  tend  to  maximize  system  parameters  vice  mission  effectiveness.  For 
example,  the  communications  panel  observed  that  if  no  system-of-system  context  is  available, 
the  tendency  is  to  maximize  bandwidth,  at  a  given  cost,  vice  acquire  communications  to  support 
overall  force  effectiveness  needs.  Consistent  with  this  observation,  the  single  system  break-out 
panels  inexorably  found  themselves  addressing  system-of-systems  issues. 

However,  as  the  system-of-systems  panel  noted,  it  is  important  to  recognize  that  there  is  a 
spectrum  of  systems-of-systems,  each  with  their  own  challenges.  To  illustrate  this  point,  note 
that  there  are  at  least  three  broad  classes  of  systems-of-systems: 

•  Planned  and  managed  systems-of-systems.  Examples  include  the  USA’s  FCS  and  the  efforts  of 
the  Missile  Defense  Agency  (MDA).  In  fact,  these  two  cases  have  appreciable  differences  due  to 
variations  in  program  control  and  funding. 

•  Oversight  of  ad  hoc  systems-of-systems.  As  an  example,  the  USN  has  established  the 
Distributed  Engineering  Plant  (DEP)  to  ensure  that  ad  hoc  mixes  of  assembled  systems  are 
mutually  supportive  and  interoperable  prior  to  going  to  sea. 

•  Ad  hoc  systems-of-systems.  This  is  representative  of  the  vast  number  of  cases  that  confront  us. 
Thus,  we  often  are  strongly  challenged  when  we  attempt  to  assemble  and  deploy  a  system-of- 
systems  to  meet  time  urgent  operational  needs. 

These  observations  led  to  the  following  conclusion.  The  image  of  the  blind  men  and  the  elephant 
has  often  been  used  to  convey  the  confusion  that  ensues  when  “stovepiped”  entities  ineffectively 
communicate  in  an  endeavor  that  requires  close  coordination.  That  image  has  been  particularly 
apt  for  the  system  acquisition  process  where  many  different  functional  communities  participate 
(e.g.,  concept  developers,  operational  users,  system  developers,  testers,  trainers,  maintainers), 
with  different  perspectives,  and  limited  communication. 

However,  that  image  may  be  misleading  as  we  conceptualize  future  acquisitions.  We  can  convey 
the  need  to  change  the  image  by  observing  that  we  have  bad  news  and  worse  news.  The  bad 
news  is  that  the  participants  in  the  acquisition  process  are  still  “blind”.  Even  though  there  have 


been  efforts  to  create  Integrated  Product  Teams  to  enhance  communications  among  the 
functional  participants  in  an  acquisition,  they  have  achieved  only  marginal  success  (due  in  part  to 
their  failure  to  share  and  evolve  key  tools  and  data). 

The  worse  news  is  that  the  “elephant”  has  become  a  “herd  of  elephants”!  As  we  realize  that  the 
real  challenge  is  to  acquire  systems-of-systems,  we  find  that  we  have  distributed  teams  of  “blind 
men”,  clustered  around  individual  “elephants”,  who  can  neither  communicate  effectively  across 
functions  or  across  systems.  This  is  a  first  order  issue  which  will  require  changes  in  culture, 
organization,  education  and  training,  processes,  tools,  and  infrastructure. 

In  addition,  it  must  be  understood  that  future  systems-of-systems  will  require  us  to  cope  with 
many  alternative  perspectives  reflecting  differing  institutional  cultures.  In  this  workshop, 
differences  were  apparent  in  two  dimensions:  across  the  Service  participants  and  between  the 
DoD  and  IC  communities.  In  the  latter  case,  there  were  distinctive  differences  voiced  with 
respect  to  classification  issues.  Ultimately,  as  we  deal  with  the  full  inter-agency,  international 
problem,  those  cultural  barriers  will  be  even  more  pronounced.  Although  this  workshop  did  not 
focus  on  the  cultural  dimension  of  the  problem,  it  is  important  to  acknowledge  its  existence  and 
to  recognize  that  a  successful  solution  must  confront  it. 


Figure  2.  Framework  for  Synthesis 


Figure  2  provides  a  framework  that  was  created  to  identify  a  set  of  categories  that  must  be 
addressed  by  the  acquisition  community.  It  is  concluded  that  if  future  C4ISR  assessments  are  to 
be  planned  and  conducted  successfully  in  a  dynamic  environment  and  supported  effectively  by 
the  M&S  community,  we  must  consistently  address  all  of  these  factors  in  a  balanced  way.  The 
backdrop  for  these  factors  is  set  by  the  cultures  of  the  many  communities  that  must  participate  in 
future  C4ISR  acquisitions.  To  remind  us  of  the  importance  of  the  cultural  dimension,  one  facet 
of  the  pyramid  is  labeled  “change  in  attitude,  culture”. 

The  base  of  the  framework  rests  on  foundation  data  and  information.  It  subsumes  repositories  of 
critically  needed  information  (e.g.,  environmental  data).  Building  on  that  foundation  are  a  set  of 
broadly  needed  support  tools.  These  include  readily  tailorable  and  reusable  scenarios.  Resting  on 
these  support  tools,  are  sets  of  hierarchies  of  measures  of  merit  (MoMs).  These  range  from 
measures  of  C4ISR  system  performance  (e.g.,  bandwidth  of  communications  systems)  to  overall 
measures  of  campaign  effectiveness  (e.g.,  the  time  required  to  halt  an  invading  army).  Once  the 
MoMs  have  been  identified,  it  serves  to  clarify  the  individual  M&S  modules  that  are  needed  to 
evaluate  the  measures  of  interest.  These  M&S  modules  are  then  federated  into  environments  that 
enable  the  acquirer  to  assess  the  measures  efficiently  over  the  requisite  set  of  scenarios  and 
assumptions.  In  order  to  develop  and  employ  these  environments,  it  is  necessary  to  have  sound 
management  and  associated  processes.  Finally,  at  the  apex  of  the  pyramid,  are  sound  policies 
that  guide  the  acquirer  in  applying  M&S  appropriately  in  the  acquisition  process. 
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Figure  3.  Synthesis  Framework:  High  Priority  Needs 


Each  of  the  panels  was  requested  to  identify  their  highest  priority  needs,  across  all  of  the  phases 
of  the  acquisition  process.  Figure  3  indicates  the  categories  for  those  highest  priority  needs,  by 
panel. 


Even  at  this  level  of  abstraction,  an  interesting  trend  appears.  Two  of  the  single  system  panels 
(sensors  and  communications)  focused  on  the  categories  associated  with  the  lower  part  of  the 
pyramid  (i.e.,  environments  down  to  foundation  data/information).  Although  the  details  of  their 
needs  varied  to  reflect  their  domains  of  interest,  they  tended  to  be  consistent  in  tone  and 
emphasis.  Conversely,  the  C2/IP  and  system-of-systems  panels,  which  tended  to  be  broader  in 
scope,  reflected  increased  interest  and  concern  about  issues  of  policy,  management,  and  process. 

The  material  that  follows,  briefly  summarizes  the  major  needs  that  were  identified,  by  category, 
and  discusses  the  similarities  and  differences  among  the  findings  of  the  individual  panels. 

In  the  area  of  policy,  two  broad  needs  were  identified.  First,  the  system-of-systems  panel  called 
for  a  revisit  of  DoDD  5000  to  ensure  that  it  addressed  critical  system-of-systems  issues 
adequately.  For  example,  there  was  a  need  to  address  mission  capabilities  based  decision-making 
vice  individual  system  decision  making.  Second,  there  is  a  need  to  develop  and  provide 
government/contractor  access  to  authoritative  M&S  and  repositories  for  data,  algorithms,  joint 
scenarios,  and  synthetic  natural  environments.  This  access  might  be  implemented  by  the 
SIPRnet  and  NIPRnet. 

In  the  area  of  management/process,  the  system-of-systems  panel  called  out  the  need  to  derive 
insights  from  system-of-systems  events  (e.g.,  exercises,  experiments).  In  order  to  do  so,  there  is  a 
need  for  sufficient  funding,  authority,  and  responsibility  to  capture  and  exploit  important  data 
from  these  events. 

Each  of  the  panels  identified  major  needs  in  the  area  of  acquisition  environments.  The  broadest 
need  was  articulated  by  the  system-of-systems  panel  which  called  out  the  need  for  secure, 
distributed,  scalable,  responsive,  standards-based,  collaborative  engineering  environments. 
Specific  attributes  of  these  environments  were  identified  by  the  other  panels.  For  example,  the 
C2  panel  called  out  the  need  for  these  environments  to  be  interoperable  with  those  of  industry 
and  to  evolve  throughout  the  acquisition  process.  The  remaining  panels  noted  the  need  for  these 
environments  to  be  able  to  interface  with  other  systems,  M&S,  humans  in  the  loop,  and  hardware 
in  the  loop. 

Each  of  the  panels  identified  major  needs  in  the  area  of  M&S.  At  a  macro-level,  the  system-of- 
systems  panel  called  out  the  need  for  a  reference  model  for  alternative  levels  of  interest  (e.g., 
system,  function,  mission,  campaign)  that  would  help  identify  key  M&S  capabilities  and 
shortfalls.  Several  of  the  other  panels  identified  specific  M&S  that  were  needed.  These  included 
communications  models  appropriate  for  system  of  systems  level  analyses  and  network  operations 
support,  planning,  and  training;  high  level  M&S  that  could  support  quick  turn-around 
assessments  for  sensor  trade  studies;  the  ability  to  model  information  infrastructures;  and  M&S 
that  featured  better  representations  of  decision  making  processes  in  C2. 

There  was  general  agreement  that  a  hierarchy  of  MoMs  is  needed  that  would  support 
assessments  of  the  impact  of  C4ISR  systems  on  mission  effectiveness.  In  the  discussion,  it  was 
noted  that  NATO’s  evolving  Code  of  Best  Practice  for  C2  Assessment  is  an  excellent  point  of 
departure  to  pursue  the  development  of  those  MoMs  (Reference  3). 


Several  panels  identified  the  need  for  support  tools  to  enhance  the  ease  of  use  and  responsiveness 
of  M&S.  In  particular,  it  was  stated  that  there  is  a  need  for  reusable,  tailorable  scenarios;  a  joint 
library  with  an  extensive  scenario  set  (including  Blue  force  laydowns);  and  common 
environmental  representations. 

Finally,  several  of  the  panels  identified  the  need  for  key  foundation  data  and  information.  From  a 
system-of-systems  perspective,  it  was  observed  that  there  is  a  need  for  synchronization  points  for 
past,  current,  and  future  system  performance  data.  In  addition,  needs  were  identified  for  common 
standards  for  inputs  to  drive  models  and  the  identification  of  architectural  data  to  link 
architectural  representations  to  executable  simulations. 

F.  Summary 

This  final  section  of  the  report  identifies  what  we  were  able  to  achieve  at  the  workshop  and 
several  residual  challenges  that  remain  to  be  addressed.  It  identifies  a  balanced  set  of  initiatives 
that  should  be  pursued  to  ameliorate  the  major  high  priority  needs  that  were  identified  by  the 
break-out  groups.  The  section  concludes  with  a  brief  statement  of  the  “bottom  line”. 

The  workshop  was  characterized  by  three  major  achievements.  First,  it  provided  a  forum  at 
which  effective  interchange  among  government,  industry,  and  academia  was  achieved.  Second,  it 
served  to  clarify  the  nature  of  the  problem  associated  with  the  M&S  needed  to  support  the 
acquisition  and  transformation  of  C4ISR  systems.  In  particular,  a  substantial  number  of  key 
needs  were  identified  and  their  relative  significance  was  assessed.  Finally,  the  workshop 
identified  a  set  of  the  most  critical  needs,  documented  them,  and  formulated  options  to 
ameliorate  them. 

Due  to  time  limitations,  the  participants  were  directed  to  focus  their  attention  on  a  relatively 
small  set  of  highest  priority  needs.  Each  panel  identified  a  much  larger  set  of  needs,  many  of 
which  warrant  additional  attention.  Those  needs  should  be  re-examined  in  subsequent 
workshops. 

In  addition,  the  workshop  was  scoped  to  restrict  attention  to  the  phases  of  the  acquisition  process 
and  activities  limited  to  DoD  and  the  IC.  As  noted  above,  it  is  important  to  identify  and  address 
the  needs  associated  with  pre-acquisition  activities  (e.g.,  the  requirements  process).  In  addition, 
US  military  operations  are  increasingly  dependent  on  inter-agency  participants  and 
multinational/coalition  partners.  Further  assessments  are  required  to  identify  the  M&S  needs  that 
their  participation  requires. 

In  order  to  support  the  acquisition  and  transformation  of  C4ISR  systems,  it  will  be  necessary  to 
undertake  a  balanced  set  of  initiatives.  This  requires  actions  in  each  of  the  categories  that 
comprise  the  framework  pyramid. 

The  items  cited  below  reflect  the  highest  priority  needs  identified  by  the  workshop  participants. 
It  is  believed  that  they  constitute  a  necessary  but  not  sufficient  set  of  actions  to  redress  the  many 
needs  that  confront  the  C4ISR  acquisition  community. 


From  a  policy  perspective,  it  is  recommended  strongly  that  DoDD  5000  be  revised  to  clarify  the 
challenging  problems  associated  with  the  acquisition  of  the  several  varieties  of  systems-of- 
systems. 

From  a  management/policy  perspective,  it  is  recommended  that  steps  be  taken  to  focus  the 
acquisition  process  on  achieving  enhanced  mission  capability.  This  implies  taking  steps  to  ensure 
that  one  acquires  the  “right  thing”  as  well  as  acquiring  the  “thing  right”. 

From  an  environment  perspective,  it  is  important  to  acquire  collaborative  environments  that 
enable  acquirers  to  explore  interface  issues  with  other  systems,  hardware  in  the  loop,  software  in 
the  loop,  and  humans  in  the  loop. 

From  a  M&S  perspective,  it  is  important  to  take  inventory  of  our  M&S  strengths  and  weaknesses 
(through  a  reference  model)  and  to  take  steps  to  redress  those  weaknesses  (e.g.,  enhance  the 
representation  of  decision  making). 

From  a  support  tool  perspective,  there  is  a  need  to  create  common  environmental  representations 
and  to  assemble  libraries  of  tailorable,  reusable  scenarios. 

Finally,  from  the  perspective  of  foundation  data/information,  it  is  important  to  provide  easily 
accessible,  broadly  available  data  on  architectures,  systems  performance,  and  organizational 
behavior. 

This  workshop  has  a  clear  bottom  line.  The  workshop  participants  are  in  broad  agreement  that  to 
support  the  transformation  of  the  C4ISR  system,  all  acquisitions  must  be  conducted  in  a  system- 
of-systems  context.  Further,  to  acquire  a  system-of-systems,  it  will  require  a  strong,  integrated 
M&S  capability.  If  the  needs  identified  in  this  workshop  are  addressed  in  a  balanced  way,  we 
will  have  taken  a  major  step  towards  realizing  those  goals. 
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